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NOVEL  FAST  AND  ACCURATE  CORRELATION-TRACKING  ALGORITHM 

Wang  Tao  and  Lin  Shangqian 

Dept,  of  Technical  Physics 
Xidian  University,  Xi'an,  710071 

Abstract:  This  paper  presents  a  novel  correlation¬ 

tracking  algorithm  on  the  basis  of  the  Forward-Looking- 
Inf  rared-  Imagery  tracking  work  of  the  aiming 
compartment.  This  algorithm  adopts  the  one-dimensional 
histogram  quick  search  and  two-dimension  multistage 
sub-sample  accurate  positioning  methods,  which  make  it 
possible  to  realize  real-time  processing.  It  has  also 
effectively  improved  the  system's  tracking  precision  by 
frame  jump  modifying  of  cumulative  errors  with  a 
sequence  of  images.  Finally,  this  paper  gives  some 
experimental  results  which  are  in  accord  with 
theoretical  results . 

Keywords:  correlation-matching,  forward-looking- 

infrared,  imaging  tracking. 

The  correlation-tracking  algorithm  has  many  unique 
advantages,  namely:  moderate  requirements  for  image  quality  and 
normal  operation  with  low  signal-noise  ratio;  simple  operation 
not  requiring  complex  image  preprocessing;  correlation  operations 
by  directly  using  image-element  gray-scale  characteristics; 
adaptability  to  a  target  and  background  with  complex  structure; 
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high  anti-local  jamming  capabilities  and  insensitivity  to  local 
modification  of  the  target  and  discontinuous  portions  of  scenery. 
Because  of  the  above-listed  advantages,  the  correlation-tracking 
algorithm  has  a  significant  place  in  image-tracking  technology 
and  cannot  be  substituted  by  any  other  algorithm.  For  instance 
in  the  initial  phase — the  search  positioning  phase — of  the 
forward-looking-infrared  image  tracking,  the  image  has  low 
contrast  and  it  is  difficult  to  separate  man-made  ground  targets 
(such  as  buildings,  bridges  and  airports,  etc.)  from  a  complex 
background  through  the  image.  In  this  case  it  is  desirable  to 
adopt  correlation  tracking  as  the  optimal  scheme,  and  an 
optimized  correlation-tracking  algorithm  appears  to  be  a  key  to 
realizing  this  scheme. 

There  are  two  major  problems  in  realizing  a  general 
correlation-tracking  algorithm  in  engineering  projects.  One  of 
them  is  that  it  requires  an  excessively  large  computational  load 
and  time,  which  is  unfavorable  for  real-time  realization.  The 
other  problem  is  that  it  is  accompanied  with  a  cumulative  error 
process,  which  may  lead  to  reference-window  drift  and  missing  the 
target.  Thus  far,  several  practical  algorithms  have  been 
created,  including  the  SSDA  method,  gray  scale  variation 
correlation  method,  etc.,  which  can  somewhat  reduce  the  total 
computational  load  but  still  cannot  meet  the  requirements  of 
real-time  image  tracking  in  modern  weaponry. 

In  this  paper  a  novel  fast  and  accurate  correlation-tracking 
algorithm  is  described,  which  was  developed  for  the  forward¬ 
looking-infrared  image  tracking  system  in  response  to  the  working 
state  and  tactical  requirements  of  the  aiming  compartments.  This 
algorithm  uses  a  scheme  of  one-dimensional  histogram  quick 
search,  and  two-dimensional  multistage  accurate  sub-sample 
positioning,  to  effectively  compress  image  processing  data  and 
provide  conditions  for  its  real-time  realization.  Apart  from 
this,  the  algorithm,  employing  the  degree  of  correlation  of  the 
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image  sequence,  adopts  a  reference-window  location  frame- jump 
correction  method  to  make  corrections  to  the  cumulative  tracking 
errors  and  raise  the  tracking  precision  of  the  system. 

1.  One-dimensional  Gray  Scale  Histogram  Fast  Rough  Search 
Algorithm 

In  the  initial  phase  of  tracking,  the  tracking  contrast 
appears  stable  and  imaging  conditions  also  remain  unchanged 
because  the  target  is  at  a  short  distance.  It  is  therefore 
believed  that  the  target  area  and  its  surrounding  image  element 
gray  scale  distribution  characteristics  can  basically  maintain 
unchanged  in  the  real-time  image  sequence,  i.e.,  in  several 
frames,  the  reference  image  containing  the  target  and  the  sub¬ 
image  located  at  the  matching  position  have  a  similar  statistical 
characteristic  height  of  the  gray  scale  histogram.  Based  on 
this,  a  one-dimensional  gray  scale  histogram  can  be  designed  to 
quickly  screen  the  matching  correlation  points.  The  specified 
steps  in  realizing  this  algorithm  are  as  follows: 

First  step:  calculate  the  gray-scale  histogram  function  of 
the  reference  image  Hs(x),  where  x  is  the  gray-scale  modification 
scope,  which  is  evaluated  as  0<x<255. 

Second  step:  select  a  sub-image  the  same  size  as  the 
reference  window  at  the  real-time  image  search  location  (n,  v) 
and  calculate  its  histogram  expressed  as  H„^^(x). 

Third  step:  calculate  the  similarity  between  Hs(x)  and 
Hu,,(x)  simply  with  the  one-dimensional  MAD  algorithm 

/;?,('/./•)  =  |]l//,(i)  -  w.,.,(i)l 

r  *0 


For  better  reliability  of  the  correlation  search,  a 
deviation  of  ±1  gray  scale  is  allowed  between  the  real-time 
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pattern  and  the  reference  pattern  plate.  Let  Hg  move  1  gray¬ 
scale  relative  to  conduct  to  subtract  correlation,  i.e.. 


2h4 


DpOi.r)  =  +  1)  -  H.Ax)\ 

(2) 

1-0 
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DT/'Oi.r)  =  1  //,(.,  -  ])  -  //,  „(j:)  1 

(3) 

Then  select  the  minimvim  value  from  the  calculated  values  in  the 
above  three  equations  as  the  measured  correlation  value  at  that 
point 

/;  „  fw  . ;  =  min  1  r ,,  >■ 'i  /)*'(■» 

(4) 


l)„(>i.r')  =  min 


Fourth  step;  modify  (n,  v),  i.e.,  repeat  the  second  and 
third  steps  throughout  the  correlation  search  area  (for  faster 
speed,  it  is  possible  to  select  only  the  sub-image  at  the  lattice 
points )  so  as  to  acquire  the  histogram  correlation-matching 
measured  curved  face  of  the  reference  image  and  real-time  image 
throughout  the  search  area  Dn(u,v) . 

Finally,  select  an  area  corresponding  to  the  valley  on  the 
curved  face  D„(u,v)  as  an  option  area  for  further  fine 
correlation-matching.  To  improve  reliability,  it  would  be  wise 
to  choose  more  areas  (i.e.,  locations  corresponding  to  the  second 
and  third  minimum  value  of  Dn(u,v)  4:1  as  possible  option  areas. 


Fig.  1.  4:1  Sub-sample  pattern  with  black- 
dot  image  elements  forming  pattern  A 
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2.  Multistage  Sub-sample  Correlation-matching  Algorithm 

During  the  tracking  of  ground  fixed  man-made  targets,  the 
imaging  device  moves  relative  to  the  scenery.  Viewed  from  the 
filming  plane,  the  whole  scene  is  translational.  To  estimate 
such  movement,  it  is  sufficient  to  estimate  the  movement  of  some 
of  image  elements  in  the  scene.  On  this  basis,  the  multistage 
sub-sample  correlation-matching  algorithm  can  be  adopted  for  the 
option  areas  determined  in  the  rough  search  for  faster  and  more 
accurate  matching. 

Structure  of  4:1  sub-sample  pattern: 

Fig.  1  shows  a  4:1  (image  elements)  sub-sample  pattern 
structure,  where  black  dots  represent  pattern  4,  while  the  other 
three  4:1  sub-sample  patterns  are  B,  C  and  D,  which, 
respectively,  are  made  up  of  image  elements  to  the  left,  under 
and  at  the  bottom  right  of  the  black-dot  image  elements.  These 
four  sub-sample  patterns  are  used  alternatively  across  the 
correlation  area  and  produce  a  better  effect  than  only  one  of 
them  used  alone.  The  alternation  principle  of  the  four  patterns 
is  as  follows:  if  pattern  A  is  used  at  the  location  (x,y),  then 
it  should  also  be  used  at  the  location  (x+2i^y+2j)  (i  and  j  are 
integers);  pattern  B  is  used  at  (x+2i^y+l+2j ) ;  pattern  C  is 
employed  at  (x+l+2i^y+2j;  and  pattern  D  is  used  at  (x+l+2i^y+l+2j)  . 
Similarly,  16:1  and  64:1  sub-sample  patterns  can  also  be 
constructed,  which  are  used  alternatively  at  different  locations 
and  formulate  a  multistage  sub-sample  pattern. 

The  multistage  sub-sample  fast  correlation-matching 
algorithm  operates  in  the  following  steps: 

First  step:  adopt  the  multistage  sub-sample  pattern  to 
perform  two-dimensional  correlation  in  every  correlation  window 
for  the  option  areas  selected  in  the  rough  search  algorithm  as 
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described  above.  First,  use  a  higher  level  (64:1)  sub-sample 
pattern  to  remove  most  non-matching  points,  and  those  left  are  a 
new  option  point  album.  Then,  use  the  16:1  and  4:1  patterns  to 
further  remove  non-matching  points. 

Second  step:  adopt  the  1:1  sub-sample  pattern  to  deal  with 
the  remaining  few  option  points  (usually  4),  i.e.,  all  image 
elements  in  the  window  are  involved  in  the  correlation 
operations . 

Third  step:  from  the  several  correlation  coefficients 
calculated  in  the  last  step,  select  the  locations  corresponding 
to  the  maximum  values  as  matching  points. 

The  dimensions  of  the  correlation-tracking  reference-image 
window  are  selected  based  on  target-background  characteristics 
and  correlation-matching  performance  in  integration.  In  our 
experiment,  the  reference  dimension  was  selected  as  (64x64). 

Whether  or  not  the  reference  image  needs  replacement  and  how 
to  replace  it  depends  on  the  confidence  level  of  correlation 
tracking.  In  fact,  correlation  tracking  is  to  realize  target 
tracking  by  replacing  the  reference  images  frame  by  frame.  If 
the  confidence  level  of  correlation  tracking  is  lower  than  a 
particular  threshold  (T,.),  it  indicates  that  the  present  image  is 
unreliable  and  it  has  to  be  further  correlated  with  the  original 
reference  image  or  otherwise,  it  should  be  replaced  with  the 
window  image  at  the  matching  location. 

The  confidence  level  signal  of  correlation  tracking  is 
designed  on  the  basis  of  analyzing  the  performance  of  the 
correlation-matching  algorithm  as  a  vitally  important  parameter. 
The  larger  the  matching  peak  value  and  the  larger  the  difference 
between  the  peak  value  and  the  correlation  measured  mean  values 
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of  other  non-matching  points,  the  more  reliable  the  tracking  can 
be.  Suppose  the  fourth  step  in  the  foregoing  correlation¬ 
matching  algorithm  generates  Q+1  measurement  i|»(J),  where  i|f(0)  is 
the  measurement  of  a  non-matching  point  while  the  rest  are  Q 
number  of  non-matching  points,  then  the  measured  mean  value  of 
the  non-matching  point  is  calculated  as  follows: 


<!>  i'J  ) 


(5) 


and  the  tracking  confidence  level  c  is 

(■  =  7’  (0)  •  [f/XO)  —  f/jJ 


3 .  Frame-Jximp  Correlation-Tracking  Cumulative-Error  Correction 
Scheme 

Because  each  matching  error  in  correlation  tracking  is  more 
than  1  image  element,  the  reference  window  may  drift  and  miss  the 
target  if  such  errors  accumulate  during  correlation  tracking.  To 
solve  the  problem,  a  frame- jump  ctunulative-error  correction 
scheme  was  adopted  in  this  paper.  The  basic  philosophy  of  this 
scheme  is  to  correct  the  cximulative  errors  generated  from  (i-L) 
frame  to  the  present  frame  (i-th  frame)  making  use  of  the  (i-L) 
frame  reference  image. 

Assume  the  corrected  reference  image  s^.l  and  the  present 
reference  image  Si.j  are  respectively  matched  with  the  real-time 
image  at  the  locations  and  ;  the  confidence 

level,  respectively,  is  f/"  and  c^;  and  the  target  image  location 
deviation  of  the  present  frame  i  relative  to  the  last  frame  (i-1) 
and  frame  (i-L),  respectively,  is  (Axi,Ayi)  and 
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(8) 
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then  the  cumulative  location  deviation  from  the  i-L  frame  to  the 
i-th  frame  is 

_  V'  \_o>  . 

(9) 


=  E  •  «„  =  E 

,)-0  J-O 


By  comparing  •  with 

if  ,  and  ,  then  the  matching  location  of  the 

present  frame  does  not  need  correction,  or  otherwise  it  can  be 
corrected  with  the  following  equation 
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(10) 
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Fig.  2.  Flow  chart  showing  correction 
of  correlation  cumulative  errors 


Key: 


(1)  Real-time  image  i 

(2)  Correct  reference  image  i-L 

(3)  Present  reference  image  i-1 

(4)  Rough  search — precise  correlation 
algorithm 

(5)  Positioning  error  of  corrected  frame 
compared  with  present  frame 

(6)  Positioning  error  of  present  frame 

(7)  Positioning  error  of  last  L-1  frame 

(8)  Calculate  cumulative  errors 

( 9 )  Judge  if  correction  is  needed 

(10)  Correct  positioning  location  of 
present  frame 
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where  L  is  the  nvunber  of  corrected  frames,  which  is  determined 
from  the  variation  of  target  dimensions,  other  distortions  and 
precision  requirements  of  the  correlation  tracking  system.  In 
our  experiment,  L=5 . 

Fig.  2  is  a  flow  chart  showing  the  process  of  correlating 
the  ciimulative  errors  of  the  correlation-tracking  system. 

4 .  Experimental  Results  and  Conclusions 

A  simulation  experiment  was  conducted  by  the  author  of  the 
novel  algorithm  proposed  in  this  paper  by  using  the  video 
pictures  of  the  forward-looking-infrared  image  filmed  on  the 
spot . 
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Fig.  4.  Photographs  of  rapid  and  precise  correlation¬ 
tracking  (positioning)  experiment 


Fig,  3  shows  correlation  curved  face  photographs  of  images 
filmed  on  the  spot,  where  (a)  is  the  MAD  two-dimensional  curved 
face  of  an  input  image;  and  (b)  is  the  histogram  one-dimensional 
MAD  correlation  curved  face  of  the  same  input  image.  It  is  known 
through  a  comparison  between  the  two  photographs  that  they  are 
extremely  similar  in  shape,  although  (b)  is  less  fine  than  (a). 
This  suggests  that  the  scheme  of  conducting  a  one-dimensional 
histogram  correlation  quick  search  of  the  lattice  is  feasible. 

Fig.  4  shows  the  multistage  sub-sample  quick  and  precise 
correlation  positioning  photographs.  The  one  on  the  top  left  is 
a  reference  image,  from  which  the  image  in  the  window  is  selected 
as  a  reference  image.  The  matching  location  is  searched  in  the 
second  image  (bottom  right)  filmed  0.5s  later  with  its  reference 
image  window  dimensions  64x64,  while  the  large  window  is  the 
search  area.  The  photograph  on  the  bottom  left  is  a  histogram 
one-dimensional  MAD  rough  search  correlation  curved  face.  In  the 
top  right  photograph,  the  cross  marks  the  location  of  precise 
correlation-matching.  The  experimental  results  indicate  that 
although  the  target  image  has  a  complex  structure  and  there  is  a 


certain  modification  of  the  perspective  angle  between  the  two 
images,  the  above-described  fast  correlation  algorithm  still  can 
realize  precise  target  positioning  to  confirm  the  effectiveness 
of  this  algorithm. 

This  paper  was  received  on  August  21,  1994. 
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